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Remarks 

I. Claim Rejections Under 35 U.S.C. SI 12, first paragraph 

Claims 1-5 were rejected under 35 U.S.C. §1 12, first paragraph, as failing to comply with 
the written description requirement. More specifically, the Action asserts that there is not 
enough evidence in the specification which clearly supports the "without generating computer 
code" claim language because one skilled in the art would understand that there must be computer 
code running in the processor, database, and computer to process the GUI haptic application as 
claimed by the Applicants. 

Applicants respectfully disagree, but have amended claims 1 and 3 to read "...without 
writing any computer code..." as a substitute for the term "generating" in order to clarify the 
intention of the claim. Applicants recognize that there must be computer code running in the 
processor, database, and computer to process the GUI haptic application. What was intended is 
that the user is not required to write any code, from a software development point of view, to 
make any three-dimensional virtual objects (or models) haptic in any PC based environment. For 
example, if a user had only a three-dimensional model of a sphere, without the presently claimed 
graphics to haptic (G 2 H) invention, one would need to write or modify the existing visualization 
code for the sphere to make it touchable. 

G 2 H allows generation of a user selected size and shape of a three-dimensional model to 
be used as a proxy in order to represent the haptic device in the computational virtual space. This 
is important since otherwise there is no way of seeing what is being touched in the physical 
virtual device workspace by the haptic device. This is a crucial mapping of the correspondence 
needed between the physical virtual device workspace in which the haptic device is physically 
moved and the computational virtual space containing the virtual scene with virtual objects. From 
software development point of view, this means that the graphical representation of the cursor 
also needs to be addressed. This would require additional code-writing if the presently claimed 
G2H invention were not used. In fact, the scene graph (containing objects) with the cursor and 
the tissue textures applied to the objects all require additional programming to generate the new 
haptic virtual environment, even if the graphical environment is already present. This is, in fact, 
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one of the fundamental differences between the presently claimed invention and the cited Rice 1 
patent. Rice actually demonstrates the need for developing haptic applications. Rice teaches a 
solution involving writing code, in conjunction with SGI based VrTools, in order to add haptic 
capability to individual anatomical models. This is a tedious approach in which each model must 
be considered separately. Rice does not propose a general software tool to convert any model 
into a haptic model. It apparently did not occur to that inventor that this could be done, or it was 
considered too difficult to implement, even in principle. This omission supports Applicants 1 
arguments in favor of novelty and non-obviousness. Once the models were available, the use of 
the presently claimed G 2 H invention would have added haptic capabilities without the need for 
any programming or any other tools. 

In light of the foregoing amendment and explanation, Applicants respectfully submit that 
it has been demonstrated that Applicants have described in the specification the claimed 
invention including, more specifically, the limitation "...without [the user] writing any computer 
code..". Thus, Applicants request reconsideration and withdrawal of the rejections of claims 1-5 
made under 35 USC §112, first paragraph. 

II. Claim Rejections Based Upon Cited References 

(a) Claims 1-5 were rejected under 35 U.S.C. § 102(e) as anticipated by or, in the alternative, 
under 35 U.S.C. § 103(a) as obvious over Rice. 

(b) Applicants note, as a first matter, that Rice was issued October 30, 2001, after the 28 
April 2001 filing of the present application, and after the 28 April 2000 filing date of the 
provisional application (Ser. No. 60/200,407) to which the present application claims priority. 
Thus, Applicants reserve the right to "swear back" of Rice in an affidavit under 37 CFR §1.131. 



1 U.S. Patent No. 6,310,619 issued 30 October 2001 to Robert W. Rice 
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(c) Before addressing specific claims, Applicants wish to describe several important 
distinctions between the presently claimed invention and Rice. 

The presently claimed invention relates to software development of general stereoscopic 
haptic virtual environments, which can include creation or importation of a three-dimensional 
graphical virtual environments, generation of the left-eye, right-eye image pairs for the view plane 
as the graphical virtual environment is manipulated (e.g. rotating or touching objects within the 
scene graph), and interfacing the stereoscopic graphics virtual environment to shutter glasses for 
stereoscopic viewing and haptic interface devices for touching objects in such a way that the 
depth perception for stereoscopic viewing can be changed for individual users and the physical 
properties of the objects can be modified as needed, e.g. soft objects can be made hard and vice 
versa. Thus, by coupling force-feedback device with a stereoscopic display, the presently 
claimed invention makes it possible to create dynamic stereoscopic-haptic immersive virtual 
environment that allows seeing the three-dimensional virtual objects with their depth while 
touching them and feeling their physical properties. 

A major differentiating component of Applicants' invention relates to the way the tactile 
component is addressed. In Rice, the sensory output is at a point of attachment of the haptic 
device to the virtual user (Col. 1 1 lines 37-40), due to the virtual user colliding with an object in the 
virtual scene because of motion. Thus, the sensory outputs felt at a point of attachment are due to 
collision. This is not the same level of tactile sensory output that Applicants 1 haptic devices are 
capable of producing when a real user is moving a haptic device in order to touch objects in the 
virtual space and actually feeling different shapes and tactile sensations of these objects. 
Computationally these two scenarios present two very different levels of development difficulties. 
When a virtual object is being touched by a real user holding a haptic device (as is the case in 
Applicants' invention) in a three-dimensional physical work space of the haptic device, additional 
levels of software development difficulties for the haptic component occurs. Examples of these 
difficulties include 1) representation of the haptic device and its motion using a proxy or a cursor in 
the computational virtual space, and 2) dealing with the stability (see the haptic stability notes 
below) of haptic applications. Rice addresses neither of these issues. 
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The presently claimed invention (hereinafter, the" G2H" system or invention) permits 
generation of a user selected size and shape of a three-dimensional model to be used as a proxy in 
order to represent the haptic device in the computational virtual space. This is important since 
otherwise there is no way of seeing what is being touched in the physical virtual device workspace 
by the haptic device. This is a crucial mapping of the correspondence needed between the physical 
virtual device workspace in which the haptic device is physically moved and the computational 
virtual space containing the virtual scene with virtual objects. From software development point of 
view this means that the graphical representation of the cursor also needs to be addressed. This 
would require additional code if G2H is not used. In fact, the scene graph (containing objects) with 
the cursor and the tissue textures applied to the objects all require additional programming to 
generate the new haptic virtual environment, even if the graphical environment is already present. 
This is one of the fundamental differences between Rice's teachings and Applicants 1 claimed 
invention. 

Regardin g haptic stability , if the haptic rendering update rates are not within the prescribed 
time constraints for the haptic device, the haptic application becomes unstable. This directly 
affects the "size" of a scene graph that can be touched in real-time. When G 2 H is used to endow the 
graphics scene with the haptic attributes, the stability of the haptic interactions are automatically 
tested and confirmed. 

The presently claimed invention comprises a tool that deals with the software development 
process, and thus it is not an application at a general user level. Rice relates to software developers, 
developing haptic applications, in contrast to G 2 H, which is an open platform for developing 
haptic applications. Rice f s system allows a user to incorporate and modify tissue-specific 
Newtonian and Biological attributes for a scene graph containing over 200 existing three- 
dimensional reduced and optimized models (based on Visible Human dataset - col. 5, 11. 13-16 and 
created by Visible Productions, L.L.C - col. 5, 11. 36-40) for simulating and animating realistic 
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human motion. All the attributes (including haptic) are used for controlling the animation of human 
motion (claims 6-8 and Col. 4 lines 4-7). 

The presently claimed invention uses, in one embodiment, 3D Studio Max to model (or 
import) any three-dimensional graphics virtual (animate or inanimate) environment, and then use 
G 2 H to incorporate and modify biomechanical tissue-specific attributes of the virtual objects to 
make them globally (include all the objects in the scene graph) touchable. Clearly, here the haptic 
device is used as a sensory output device. In this respect, the Haptic Virtual application developers 
using G 2 H do not need to do any additional programming once they have a graphical virtual 
environment. This includes being able to apply and change the sense of touch in real-time (e.g. 
making hard objects soft or vice-versa) using haptic textures that can be created, modified, saved, 
and applied to any object without any programming. G 2 H also allows generation (or importation) 
of different model types, e.g. one-layered or multi-layered poly-mesh, and parametric. 
Furthermore, a physical model can be converted to a virtual three-dimensional model and then make 
it touchable. The newly generated haptic virtual environment is then used as a foundation for 
developing user level applications. Providing the stability for haptic interactions and touching the 
inner layers of virtual objects are some of the most difficult tasks for software developers. In 
support of this, working with G 2 H also allows for interactive testing and modifications of the 
developed new haptic application. Thus, one of the most important benefits is that testing using 
G 2 H assures the largest possible stable haptic scene that can be incorporated in an application. 

(d) With regard to claim 1, the Action asserts that Rice anticipates or renders obvious all the 
limitations of the claim. Rice's system is specific to the SGI work-station (col. 5, line 38) and the 
SGI based Virtual Reality Tools. This is in contrast the presently claimed invention, drawn to 
use of actual tactile sense of touch in virtual environments (such as surgical simulation) and not 
solely based on existing models. In fact, one can generate or import any model (in existing 
standard formats) into the PC-based system, making them accessible to more people, and one can 
even use a physical model to generate a virtual model 
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Rice describes animation of movement of body structures (only poly-mesh models), and 
does not teach or suggest the ability to perform surgical simulations. 

(dl) The Action asserts that the claim 1 limitation means for providing a cursor with 
attributes of movement within multiple layers of a graphic display to create or modify one or more 
virtual object is taught in Rice at: col. 3, 11. 1-8; col. 4, 11. 15-20; and col. 6, 11. 10-18. Applicants 
respectfully disagree that these or any part of Rice teach or suggest cursor generation capability 
by a user of the application. In fact, nowhere in Rice is this capability is noted. The G2H 
invention allows generation of any shape of a three-dimensional model to be used as a cursor in 
order to represent it as the haptic device for the newly generated haptic virtual environment. This 
is one of the fundamental differences between Rice's teachings and the presently claimed 
invention. Rice's lack of a proxy implies that they do not need to monitor the movements of the 
haptic device in the device work space or the computational work space, which would actually be 
needed if the objects are touched for the texture tactile output. This is precisely what G 2 H 
provides when any three-dimensional graphical models (animate or inanimate) are converted into 
haptic models. 

(d2) The Action asserts that the claim 1 limitation means for generating a haptic 
representation of said one or more virtual objects directly from a graphical representation of said 
one or more virtual objects, wherein said one or more virtual objects comprise a plurality of 
layers that are represented by a three-dimensonal poly-mesh form is taught at col. 6, 11. 9-48 in 
Rice. Rice describes use of grouped single-layer poly-mesh models that behave as a unit for 
human motions (col. 6, 11. 49-61), but these models do not have layers. For haptic applications, 
the ability to touch the internal layers within a poly-mesh surface based model is never implied. 
Technically, this is an important contribution of the presently claimed invention. The cited Rice 
reference deals with the reduction and the optimization of number of polygons in single-layer 
poly-mesh models in order to be able to graphically render and visualize the scene graph in real- 
time while preserving all model features. There is no mention of haptics in these lines or that the 
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models are comprised of layers. This deals purely with the graphical single layer poly-mesh 
scene graph objects, not with haptics. 

(d3) The Action asserts that the claim 1 limitation means for creating, modifying, and saving 
haptic properties of said one or more virtual objects for creating a heuristic database and 
creating or modifying such a heuristic database is shown inherently by Rice's use of the user- 
variable tissue-specific attribute data relating to the nature of tissues. Rice computationally 
associates the attributes with objects that can be modified, as opposed to the inventive G2H 
system, which uses haptic textures with attributes (physical, biomechanical, or any possible 
other applicable characteristic or attributes) that can be applied and modified to any models. In 
fact, such textures are saved in a library of textures to be applied once the models are made haptic 
using G 2 H. The portions of Rice cited in the Action (col. 3. 11. 1 1-16; col. 4, 11. 5-20; and col. 10, 
11. 5-35) refer to the tissue-specific attributes regarding the animation of human motion (see col. 3, 
line 11; col. 4, line 6; and col. 10, 11. 3-4). This does not imply that the models will become 
globally touchable using a haptic device as an output device and can be used in medical 
applications, in general. 

With regard to saving the setup into data files, col. 8, 11. 39-65 of Rice refer to the possible 
transformations. For translation, it provides axis position of the node relative to the parent node 
in inches and for rotation it provides angle to be set for the node around the X/Y/Z axis relative to 
the parent node in degrees. Also provided is a time interval between the two movement positions 
in seconds. This information relates to the animation of the human motion and does not by any 
means imply creating, saving, or modifying heuristic tissue attributes database that can be used 
for touching (to feel the shape or the texture of) any virtual objects. With regard to Finally the 
last citation in the Action related to this limitation (col. 10, 11. 35-50) teaches a broad spectrum of 
possible theoretical "tissue-specific attributes". This is a conceptual generalization and there is 
no evidence in the patent that computationally this is even possible. Computational components 
of such concepts are very difficult to realize and are not trivial, as suggested by the Action. Even 
if Rice allows properties to be created, modified, saved, and associated with objects themselves, 
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it is not clear that these attributes can be transferred effortlessly (as can be done in Applicants' 
invention) to other objects (or models). 

(d4) With regard to the claim 1 limitation means for selecting all or a portion of said haptic 
properties from said heuristic database for the modeling of haptic virtual environments, the 
system as a whole being constructed and managed so that a user can create said haptic virtual 
environment without writing any computer code, references to Rice cited in the Action refer to the 
process of re-creating the 200 basic models (from the Visible Productions, L.L.C) that can be 
used for real-time animation and then scripting to actually develop the software application to 
animate the human motion. The process consists of two steps: 1) once the models are acquired 
they use commercially available SGI based virtual reality tools (PolyRed, Ez3D VRML Author, 
VrTools) to reduce, optimize, and author the models in feature invariant way (thus creating 
models that satisfy the needs of rendering them in real-time for the animation), 2) using the 
VrTools also for scripting to animate the actual human movements for various scenarios. Thus, 
Rice teaches the reduction, optimization, and the integration of the models only with respect to 
the real-time graphics rendering and visualization of the animation (col. 5, line 13 to col. 6, line 
61) and the failure to teach anything regarding the issue of haptic stability with respect to the 
haptic rendering (as well as from the size of the scene graphs seen in Figures 1 and 2, and 
hierarchical description of the tree-graph at col. 6 line 62 to col. 7, line 22) implies that the haptic 
component is "local" at the point of attachment (col. 1 1, 11. 37-40) and not "global" in the sense 
of being able to touch any object in the scene graph, as is the case of the presently claimed 
invention. Furthermore, the graphics and haptic update rates are very different, needing 
synchronization (especially with the proxy representing the haptic device) and, thus, the 
concerns with the graphics update rates do not cover the haptic update rates (which are usually 
faster by 30 times). This also requires additional programming in order to transform a graphical 
virtual environment into a haptic virtual environment. Once again, differentiating Applicants* 
invention. 
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This process basically prepares the foundation of graphical virtual environment (not 
haptic virtual environment) for generating various animations of interest. 

Rice does provide (at col. 1 1, 11. 25-62) some relevant information in the sense that the 
haptic device (and other I/O devices are) is used to input and modify the tissue-specific attributes 
(variables). As far as the properties for a tactile haptic environment is concerned, however, this is 
limited only to the collision due to motion of the virtual human to an object in the scene graph 
component. The focus of haptics in that section of Rice is for moving the body structures and 
setting the attributes that control the motion. This does not address the general tactile (touch and 
feel) haptic environment attributes G2H, which, again, focuses on creating haptic virtual 
environments for developing haptic applications. Specifically, the presently claimed invention 
deals with creation of tissue-specific tactile attributes for development of stable, haptic 
application and thus G 2 H is a development platform. The properties used in these cases are for 
controlling the sense of touch. 

In light of Rice's failure to teach or suggest, explicitly or inherently, the limitations 
discussed above, Applicants respectfully submit that claim 1, as amended, is now allowable over 
the cited art. By virtue of its dependence from claim 1, claim 2 is believed to be similarly 
allowable. 

(e) The Action asserts that Rice anticipates or renders obvious all the limitations of claim 3. 
Applicants respectfully disagree. Rice does not teach a method for developing and utilizing 
complex and precise haptic (tactile) virtual objects needed for a surgical simulation, and 
Applicants respectfully suggest that impermissible hindsight is utilized in making the deductions 
necessary to derive the presently claimed invention from the portions of Rice referred to. For 
example (as noted above), Rice does not teach creating a cursor with attributes of movement 
within multiple layers of graphics display to create or modify one or more virtual objects. Rice's 
models are single layer reduced and optimized poly-mesh single surface models. Grouping of the 
objects or models does not imply that the objects have layers. Movement within layers of a 
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tactile hap tic object with a haptic device is a sizable computational effort. The stated fact cannot 
also be deduced from the reference to col. 6, 11. 1-48. (please refer to the discussion above with 
reference to claim 1 for comments equally applicable here.) And since there is no evidence that a 
cursor exists in Rice's system, selecting an object with a cursor seems improbable. Use of a GUI 
does not imply a cursor representing a haptic device. 

In light of Rice's failure to teach or suggest, explicitly or inherently, the limitations of 
claim 3, Applicants respectfully submit that claim 3, as amended, is now allowable over the cited 
art. By virtue of their dependence from claim 3, claims 4 and 5 are believed to be similarly 
allowable. Applicants respectfully request reconsideration and withdrawal of the rejection of 
claims 3-5. 

(f) With regard to claim 5, the comments of section (d3) are incorporated here by reference. It 
is not at all clear to Applicants that the theoretical generalization asserted in the Action are 
tangible. Computationally, it is not evident that such concepts can be developed realistically in 
any software application. No evidence of such possibility is demonstrated in Rice. 
Computational components of such concepts (or even much simpler concepts) are very difficult 
to develop, if not impossible. 

In light of Rice's failure to teach or suggest, explicitly or inherently, the limitations of 
claim 5, Applicants respectfully submit that claim 5 is patentable over the cited art. Applicants 
respectfully request reconsideration and withdrawal of the rejection of this claim. 

(g) In an attempt to summarize the comments above, Applicants respectfully submit that 
some of the assertions in the Action upon which the rejections are predicated are inappropriate 
as the presently claimed invention relates to a completely different tactile haptic functionality. 
Rice considers the haptic attributes resulting from the movement of a virtual human, with 
attached haptic device, colliding with a virtual object in a scene graph. The presently claimed 
invention relates to tactile sensations of "feeling" shapes and textures of any objects in the haptic 
virtual environment. Furthermore, in addition to integrating single layer poly-mesh virtual objects 
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in the scene graphs of the inventive system, as Rice teaches, the present invention also supports 
multi-layered poly-mesh and parametric surface based models. Rice's graphical virtual 
environment consists of over 200 pre-existing surface based single layer poly-mesh models in a 
VRML format, while the instant G 2 H system has the capability of generating or importing many 
types of models (including creating virtual models from physical models) represented using the 
standard graphics formats, including VRML. 

In light of at least the foregoing, Applicants respectfully submit that claims 1-5 are now 
in a condition for allowance, and a notice to that effect is earnestly solicited. If any questions 
arise during the review of this amendment/reply, the Examiner is invited to contact John 
Hamilton at (617) 854-4000 to discuss any issue. 



Respectfully submitted, 



BHARTI TEMKIN, et al., 
Applicants 



Date: March 16, 2005 




/^fohn A. Hamilton, Reg. No. 48,946 



Attorney for Applicants 
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